On the statistical efficiency of the LMS algorithm with nonstationary inputs
نویسندگان
چکیده
A fundamental relationship exists between the quality of an adaptive solution and the amount of data used in obtaining it. Quality is defined here in terms of “misadjustment,” the ratio of the excess mean square error (mse) in an adaptive solution to the min imum possible mse. The higher the misadjustment, the lower the quality is. The quality of the exact least squares solution is compared with the quality of the solutions obtained by the orthogonalized and the conventional least mean square (LMS) algorithms with stationary and nonstationary input data. When adapting with noisy observations, a filter trained with a finite data sample using an exact least squares algorithms will have a misadjustment given by ME?, number of weights N number of training samples If the same adaptive filter were trained with a steady flow of data using an ideal “orthogonalized LMS” algorithm, the misadjustment would be M=!L= number of weights 4 Tmse number of training samples Thus, for a given time constant rmse of the learning process, the ideal orthogonalized LMS algorithm will have about as low a misadjustment as can be achieved, since this algorithm performs essentially as an exact least squares algorithm with exponential data weighting. It is well known that when rapid convergence with stationary data is required, exact least squares algorithms can in certain cases outperform the conventional Widrow-Hoff LMS algorithm. It is shown here, however, that for an important class of nonstationary problems, the misadjustment of conventional LMS is the same as that of orthogonalized LMS, which in the stationary case is shown to perform essentially as an exact least squares algorithm.
منابع مشابه
Statistical efficiency of adaptive algorithms
The statistical efficiency of a learning algorithm applied to the adaptation of a given set of variable weights is defined as the ratio of the quality of the converged solution to the amount of data used in training the weights. Statistical efficiency is computed by averaging over an ensemble of learning experiences. A high quality solution is very close to optimal, while a low quality solution...
متن کاملFrequency Estimation of Unbalanced Three-Phase Power System using a New LMS Algorithm
This paper presents a simple and easy implementable Least Mean Square (LMS) type approach for frequency estimation of three phase power system in an unbalanced condition. The proposed LMS type algorithm is based on a second order recursion for the complex voltage derived from Clarke's transformation which is proved in the paper. The proposed algorithm is real adaptive filter with real parameter...
متن کاملGenetic algorithm for Echo cancelling
In this paper, echo cancellation is done using genetic algorithm (GA). The genetic algorithm is implemented by two kinds of crossovers; heuristic and microbial. A new procedure is proposed to estimate the coefficients of adaptive filters used in echo cancellation with combination of the GA with Least-Mean-Square (LMS) method. The results are compared for various values of LMS step size and diff...
متن کاملOn the Statistical Efficiency of the LMS Family of Adaptive Algorithms
AbslrabTwo gradient descent adaptive algorithms are compared, the LMS algorithm and the LMSNewton algorithm. LMS is simple and practical, and is used in many applications worldwide. LMWewton is based on Newton's method and the LMS algorithm. LMSiNewton is optimal in the least squares sense. It maximizes the quality of its adaptive solution while minimizing the use of training dah. No other line...
متن کاملA Nonlinear Neural FIR Filter With An Adaptive Activation Function
An adaptive amplitude normalized nonlinear gradient descent (AANNGD) algorithm for the class of nonlinear finite impulse response (FIR) adaptive filters (dynamical perceptron) is introduced. This is achieved by making the amplitude of the nonlinear activation function gradient adaptive. The proposed learning algorithm is suitable for processing of nonlinear and nonstationary signals with a larg...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- IEEE Trans. Information Theory
دوره 30 شماره
صفحات -
تاریخ انتشار 1984